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ABSTRACT 



Candidates taking the Test of English as a Foreign 
are currently asked to supply several items of 



Language (TOEFL) 

background information: birth date, sex, number of times TOEFL has 
been taken before, whether they are seeking status as undergraduate 
or graduate students, their native country and native language, and 
(for prospective graduate students only) their intended field of 
study. The candidates provide this information on one side of their 
answer sheet, according to instructions given in the handbook sent to 
each candidate upon registration. In view of the extreme 
heterogeneity of the TOEFL candidate population it se^s likely that 
additional information would be valuable. Foreign studetits (39 
undergraduate and 18 graduate) were interviewed at four institutions. 
Four variables emerged as being the most promising: total number of 
years of formal study of English; reading of English-language 
newspapers, books, and magazine^; number of years of father's 
education; and number of years of mother's education. Collectively, 
these variables offer the promise of enhanced research on various 
aspects of the TOEFL candidate population and on foreign student^^ 
generally. The appendices include the interview questions used in the 
study, as well as interviewees' comments on TOEFL. (PN) 
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AN INQUIRY INTO POSSIBLE NEW ITEMS OF BACKGROUND INFORMATION 
ABOUT TOEFL CANDIDATES 

Robert A. Feldmesser 

In the conduct of a large-scale standardized testing program, It Is 
frequently useful to have Information about the characterletlcai of the 
candidates who take the test. Such Information can help pjpogram sponsors, 

/ 

test users, and the candidates make sounder Interpretations of test 
scores and test-Item responses, and It can also ale^^ program sponsors to 
needs for change In the services offered; help In^rm the educational 
community about the nature of the population talfi^ng the test; and provide 
a body of basic data to facilitate research oq^ sjuch matters as conditions 
conducive to high scores, trends In scores oVer time and among various 
groups, relationships among candidates taking several different tests, 
the flow of students Into and through educational Institutions, and other 
topics that cannot be anticipated. 

Candidates, taking the Test of Ertgllsh as a Foreign Language (TOEFL) 
are currently asked to supply several Items of background Information: 
birth date, sex, number of times TOEFL has been taken before, whether 
they are seeking status as undergraduate or graduate students, their 
native country and native language, and (for prospective graduate students 
only) their Intended field of study. The candidates provide this Informa- 
tion on one side of their answer sheet, according to Instructions given 
In the handbook sent to each candldate^upon registration. This Information 
Is often helpful for the purposes mentioned above, but — especially In 
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vlew of the extreme heterogeneity of the TOEFL candidate population 
(Wilson, forthcoming [a,b]) — It seems likely that additional Information 
would be valuable. This study was undertaken to explore the kinds of 
Information that (a) -were perceived as potentially helpful by persons 
professionally concerned with foreign students, (b) were feasible to 
obtain from foreign students, and (c) were most promising as variables to^ 
Improve the Interpretation of TOEFL scores and to enrich research on the 
candidate population.^ 

Data Collection 

The basic data-gathering procedure for the study was a set of Interviews 
with foreign students enrolled at four U.S. Institutions of higher education. 
In order to decide what the most productive questions for these Interviews 
would be, pertinent publications were examined (e.g., Althen, 1978, 1981; 
National Association for Foreign Student Affairs*, 1978, 1981; Sharp & 
others, 1971; Wray, 1981), conversations were held with members of tf^ TOEFL 
program staff, and letters were sent to various professional persons with 
special knowledge of foreign-student affairs — directors of foreign-student 
centers, professors of English as a Second Language, admissions officers and 
other administrators, and staff members of private and governmental agencies 
concerned with foreign* students. (Members of the TOEFL research committee 
and Policy Council were among those contacted.) These letters described 
the project's objectives. Indicated the kinds of questions that were being 
considered for Inclusion In the Interviews, and Invited suggestions for 



Funds for this study were provided by the research committee of the 
TOEFL program. Joan L. Borum, Wlllem C. Spits, Charles W. Stansfleld, and 
other members of the TOEFL program staff furnished Invaluable assistance at 
several points. The project also benefited from the work of Leta Davis as 
research assistant. 
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additional questions* Many suggestions were received, both for elaborations 

or refinements of questions already proposed and for new topics* On the 

basi^ of theses explorations and inquiries, a draft interview instrument was 

drawn up and sent to the same individuals and a few others, asking for 

comments about the questions and also fox, suggestions for suitable 

interview sites. .Of the 25 professionals who were sent letters in either 

the first or the second wave, or both, replies were received from 20 on 

either or both occasions* 

On the basis o4^^is' work, a "final" interview instrument was prepared — 

though it was later modified in minor ways during the (bourse of the 

interviews, as questions were found to be unproductive or as ways of 

r 

clarifying them became evident* The questions were mostly of a semi -structured 
type* Those relevant to the present study dealt with the 'type, amount, and 
place of the interviewees' education; their contact with the English language, 
both in formal educational and io other af€ttings; their experien<?^s with TOEFL; 
their knowledge of other languages; and the amount of schooling their mothers 
and fathers had had* In view of the special circumatancfes of foreign 
students, particular attention was paid, in framing the questions (^nd, 

\ 

later, in evaluating the responses), to the clarity and simplicity of the 
wording, the likely ability of, the interviewees to give reliable responses, 
and the possible sensitivities of these students* During the interviews, 
special efforts w^re made to give students ample opportunities for asking 



2 

A few questions were also asked about respondents' motives for coming to 
the United States to study and about the source and kind of information they 
had had about higher education in this country* Tljese questions were related 
to a possible future study of foreign-student applications and enrollments, 
and they will not be discudsed in this report* 
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questions and stating opinions and to provide positive feedback, In order 
to overcome any hesitation about answering (Oksenberg & others, 1977). 
The interviews rarely required morp than 30 minutes, and no serious 
difficulties were encountered in conducting them. A copy of the interview 
instrument in the last form in which it >as used is reproduced as 
Appendix A. * 

The interviews were held between October and December, 1981, at 
Princeton University, Rockland Comtaunity College, Temple University, and 
the University of Maryland. These institutions were^ selected not out of 
any effort to achieve "representativeness," but rather for the diversity of 
their student 'bodies , the closeness of their campus to the location of the 
.project offices (in order to minimize the costs of the study), and the 
availability of a person on each campus who was willing to serve as a local 
contact for the study. ^ Each contact person was aslced ta recruit' for the 
interviews about 20 students from non-English-speaking countries. They 
{ were all to have taken TOEFL within the past three years, and they were to 
1>R^dtvided about evenly betwee/i males' and females and between undergraduate 

and graduate students (if the instiCntion enrolled ^any of the latter). The 

- f ^ 

contact persons were told that we wanted to interview some students from 
each of five geographical regions—the Far East, the Middle East, Hispahic 



Administrative officials were arsd interviewed at each of 
these campuses for their views on the problems of obtaining useful and 
reliable information from foreign students. * ^ ^ 

am happy to acknowledge the indispensable assiatance of Nina Issawl, 
, foreign-student adviser, and Judith P. Mackenzie, f orelgn-atudent admission 
officer, Princeton; Jonathan Lambert, coordinator of the Center for 
International l^udents, Rockland; Margaret Tisa, academic coordinator of 
the Intensive English Language Program, Temple; and Lois Lanier, Maryland 
English Institute*. 
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America, Africa/ And^ Western Europe* The contact person was sent a supply 
of one-page descriptions of the study to be given to prospective interviewees 
so that they could make an informed decision about whether to participate. 
The description stressed the voluntary and anonymous nature of parti^cipatlon 
and the fact that neither the decision about participation nor the students' 
responses if , they did participate would have any effect on their' studlent 
status. 

The goal was to obtain interviews with a total of 75 students » but 
the goal was not met* At two of the campusesp fewer than 20 interviewees 
were recruited; the contact persons^ reported that the chief problems 
wer^ not in obtaining students consent to be interviewed but in locating 
them and in finding times that were convenient for them. At all the 
campuses y some students who were scheduled for an interview did not appear. 
Thus, by the time the data-collection phase of the study had to be ended, 
only 57 students had been interviewed. 

Characteristics of the Sample 

Of the 57 interviewees, 39 were undergraduates and 18 were graduate 
students; 37 were male and 20 were female, with the sexes distributed 
between unxlergraduates and graduates in virtually the same proportions. 
By geographic tegion, 11 came from Africa, 27 from the Far East, 7 from the 
Middle East, 3 from Europe, and 9 from Latin America. In all of these 
respects, the sample's comf^osition was remarkably similar to that of the 
entire body of foreign students in the U.S. in 1979-80, the last year for 
which such data ane available. The comparisons are shown in Table 1. 
Nevertheless, because of the way in which the sample was selected, no claim 



Table 1 

Characteristics of Sample and of All Foreign Students 

in the U.S. .. . 

ti 

.f^ All foreign students 

CharaQteristic Sample 1979-80 



Per c en t tna 1 e 




65 




72 


Percent undergraduates^ 




.68 




65 


> c 
Percent who came from: 

> 










Africa 




19 




14 ■ 


Asia (Far East & Middle Ea^t) 




60 




62 


Europe 




5 




8 


Latin America 




16 




16 


^Source: Boyan (1981), pp. 3-4, 20, 


22. 








^Among students enrolled at two-year 


and 


four-year 


institutions 





Excluding (from all foreign students) those from Canada and Oceania. 



* 
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*can be nade for Its representativeness In the statistical sense. All that 
Is claimed — and all that was sought — Is a high degree/bf diversity. Foreign 
students of many different kinds were among the Interviewees. 

Their diversity can be further seen In Table 2, which lists all of the 
countries named by one or more Interviewees a's their ^'native country." 
A comparison with the total foreign-student body Is not possible, because 
the International Institute of Education, which conducts the annual census 
.of foreign students, does not publish a comprehensive list of countries of 
origin. However, there is a point here that Is worth noting. The HE census 
asks campus officials to list the number of foreign students by "country of 
citizenship" (Boyan, 1981, p. 139). TOEFL candidates, on the other hand, 
are asked to give their "native country." Mljjdful of this difference, Ni^e 
asked Interviewees both what, they regarded as their native country and what 
their country of citizenship was. In all but eight cases, the two countries 
were the same. The except>loh8 were understandable and easily Interpretable. 
Thus, one Interviewee said Puerto Rico was her native country (and It Is 
Indeed Included as a separate entry In the list of native countries glve^ 
In the Handbook for TOEFL examinees), but she Is of course a citizen of 
the U.S. Another Insisted that Jerusalem was her native country (It Is not 
Included In the Handbook list), though she carries a Jordanian passport. 
A third referred %p his »natlve country as Namibia — the name used by many 
residents of Southwest Africa, which technically Is a possession of the 
Union of South Africa. Jhus, It would probably b<5 valid In future 
research to compare data on TOEFL candidates' native countries with data { 
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Table 2 

Interviewees* Self-Reported Native Countries 



Native country 



Number of: 



Undergraduates Graduate students 



Af ylca 

Alger la 

Ethiopia 

Ghana 

Gu In ea -Bissau 

Mauritius 

Morocco 

% a 
Namibia 

Nigeria 

Somalia 

Far East 
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China (People's Republic) 

Hong Kong 

India 

Indonesia 

Japan 

Korea 

Malaysia 

Sri Lanlca 

Thailand 

Talwarf 

Vietnam 



Middle East 



Iran 

, a 
Jerusalem 

Jordan 

Lebanon 

Saudi Arabia 

Turkey 

Europe 

Greece 
Hungary 
VJest Germany 

Latin America 

Argent Ina 
Chile ^ 
. Haiti 
Mexico 
Panama 

a 

Puerto Rico 
Venezuela " 



16 



1 
2 
2 
1 
1 
1 
1 

1 



4 
2 
1 
1 
2 
1 



1 
1 



1 
1 
1 
1 
1 



1 
1 
1 
1 

1 
1 
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.5 



1 
1 
1 
2 
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on I^E countries , of* citizenship.' 

Because their score on TOEFL was so important a variable in thiif study » 
interviewees' were askfd for permission to obtain their score from 
institutional filea^I , Most; pf them did give permission and then volunteered 

t(\eir score besfdesr, sometimes even giving their scores from several. 

' ' 6 

administrations. ( Interviewees, had taken TOEFI4 1.5 times on the average.) 

When interviewees did give permission, the contact persOo at their campus 

was requested to report the score to us, and that request was met in all 

but a few Qas'es where no^ecore could be located in the files. When 

respondents did not give permission, it was usually because they said it 

wasn't necessary since they would give us their score themselves, and they 

proceeded to do so. When Vcspondents did not give permission a^d did not 

t 

volunteer their score, they were asked if they **would mind" giving us their 
score themselves*^ -And when interviewees did give permission but did not 
volunteer their score, Jihey were asked if they "happen to remember" what 
their score was. As a result of this series of questions, we had both 
institutional and self-reported scores Lor 29 interviewees, only 
institutional scores for 3, orr^ly self ^reported scores for 16, and neither 
for 7 (af whom 2 said they had never taken TOEFL). ^ / * 
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-^In order to be sure that TOEFL candidates had a correct understanding 
of the term "native country," we asked them how they would explain it to 
another student from their own country. Virtually all of them answered 
with words like "the place where you were born," "whete you gr«W up," or 
"where your parents (or your family) lived." 

Only total scores were sought, and interviewees rarely mentioned 
subscores. ^ 

^Chiefly as a rapport-building device, we also asked respondents for 
their opinions about TOEFL as a measure of their knowledge of English. 
Although their responses are not directly relevant to this study, they 
may nevertheless be of interest and so are discussed briefly in Appendix B. 

12 ■ 



In order to learn whether we could safely use self-reported scores 
when we did not have institutional scores , we compared ' the scores of \ 
interviewees for whom we had both l^inds, using the most /recent score wher^ 
"^N^^^ there was more than one from either source. The noean difference between 
them was barely ten points; the product-moment correlation was .94. We 
concluded that the self-reported scores were fully reliable , and we have 
used them in all subsequent analyses when an institutional score was not 
available. It shoulclAbe noted^ however, that with the elimination of the 

seven candidates for whom we were unable to pbtain a TOEFL score of any kind, 

' - ■ / - ■ • , , . ' 

the effective sample was reduced to 50. The mean score earned by these 
, 50 interviewees was 52^, with a standard deviation of 69, compared to a I 
mean of 499 and a standard deviation of 67 among all^ candidates in 1978-80 
(Test of English as a Foreign Language, 1981, p. 20). The highest €core 
among the 5Q was 670, the lowest 375. ' 

■ 

Screening of Variables 

In addition to the variables already mentioned, the Interviews 
yielded information on about two dozen other variables which, were of . 
potential usefulness. They were: ' 
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This is consistent with the finding of a study by Powers (1980). 
Foreign candidates who take the Graduate Management Admission Test are 
asked to give their TOEFL score if they have taken this latter test. 
Powers was able to locate actual TOEFL scores for 2,067 candidates who 
had reported a TOEFL score upon taking the.GMAT between 1977 and 1979. 
He found the correlation between self-reported and actual scores 
to, be .91. ^ V 
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General educational back'ground , ^ 

Number of yedrs attended school In home country 

Last educational document/credential received In home country 

Number of years of study completed In the U.S. 

fype(s) of educational institution (If any) attended In the 

before enrol^llng at present institution 
Number of years of education (If any) In countries other than 
home country and U.S. ^ 

Experience with the English^ language 

Number of years of formal study of English 

Number of years of schooling In which English was the language 

11 . 



of Instruction 



9, 



Wherever variables tn this and the next category Involve numbers of 
years, the number referred to Is tpe number of years prior to taking 
TOEFL (or prior to the most recent! time it was taken, if it was taken more 
"than once). \ 

^^A few respondents had attended a program of English as a Second Language 
(ESL) in a U.S. Institution before earning their most recent TOEFL score, 
and in those cases their experience in the program was considered as part 
of their experience^ with English if the relevant information was available. 
Otherwise, however, the reference is to respondents' experience with English 
in their home country. 

^^Inf oirmation for this variable was derived from interviewees' answers 
to questions about the kin^s of schoals in which they had studied English, | 
whether their teachers spoke English in class, whether English-language 
books were used in their courses, and whether they had had any instruction 
in English in the U.S. (questions 2.1, 2.3, 2.4, 2.5, and 2.6 of the interview 
instrument). A year was counted as one in which English was the language 
of instruction only when (a) the interviewee was attending an educational 
institution in the U.S. or other English-speaking country, or (b) the 
interviewee was attending school in a non-English-speaking country 
(including, of course, the home country) but was studying a normal range 
of courses all of which were taught in English. \ 



Number of hours per week in English class during last few 

years of schooling 
Whether interviewee had had a job where English was used 
Wl^ether any members of interviewee's family knew English 
wiiether interviewee had had a^y friends who knew English 
Whether interviewee had read English-language newspapers, 

magazines, or books (other than in school); and if so, 

how often x 

/ 

( ' Whether, interviewee had listened to English-language radio or 
television programs or had seen English-language films; 
and if so, how often •« 
Whether interviewee had been in the U*S* or any other English- 
speaking country before enrolling in the institution 
presently being attended 
Knowledge of other languages 

Interviewee's native language 

Languages other than native language and English that interviewee 
knew, and how they had been learned . 
^ Which languages respondents rated as their strongest and 

second strongest 
Family background 

Whether any brothers or sisters had attended school outside their 

native country 
Country in which interviewee's father grew up 
Number of years of father's schooling 
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CcHintry in which interviewee's mother grew up 

Niunber of years of mother's schooling 
These, then^ were the variables ''from among which selection would be 
made for recommendation to be included among the informational items to be 
obtained from TOEFL candidates in the future, in addition to those items 
already being obtained. It was clear from conversations with the TOEFL 
program staff t|iat not more than three or four new items could^ be added, 
because of limitations on the space available , oh the test's answer sheet 
(and the probably unacceptably large costs that would be entailed by any 
expansion of tile answer sheet }• For purposes of this study, it was decided 
that, to be recommended for inclusion, an item should meet the following 
criteria: Cl) It should lead to the creation of two or more reasonably 
well-populated categories (rather than to ^a set in which most or all 
candidates fell into the same category). (2) It should be readily 
understood ^nd reliably answer-ed by candidates fi:om a wide variety of 
cultures, and it should not be' offensive to any of them. (3) It should 
yield information on characteristics that would be useful in the interpre- 
tation of TOEFL scores in an institutional .^election process. ''Useful" 
characteristics would be those that were related to the scores in ways 
that were both plausible and statistically promising. Within this 
constraint, extra consideration would be given to characteristics that 
had a high probability of enhancing other kinds of research on TOEFL 
candidates and on foreign students generally. 

Many of the variables failed to meet the first criterion. Very few of 
the interviewees had completed more than one year of study in the U.S. 
before taking TOEFL, even fewer had been in the U.S. before that for other 
purposes, and almost as , few had received any education in countries beside 
the U.S. and their ndtivl^ country (and in nearly every case for less than a 

\ 16 



year). On' the other hand, the great majority of interviewees said that some 

or all of the other members of their family knew some English, and it did .not 

seem feasible to distiilguish among them on the basis of how many (or w^at 

proportion) of them knew English, much less on the basis of how well they 

knew it. All' of the respondents had some friends who knew English, 

simply because they had all studied English in school and so they had 

classmates who knew English — though most interviewees said they had never 

used English with their friends outside the classroom. All but a few 

listened frequently or regularly to English-language radio and/or TV. 

The fathers and mothers of nearly all the interviewees had grown up in the 

same country a^ their children. , 

TOEFL candidates are already asked for their native language^ so there 

12 

is no need to consider it as a possible additional item. About 60 percent 
of the interviewees said they knew at least one other language; the "other" 
languages spoken by the interviewees of each native language are shown in"" 
Table 3. The variety of these other languages, the number of different 
combinations with native languages, and the variations in degree of fluency 
are so great that this could hardly be made into an interpretable variable. 
Most interviewees said that their native language was their strongest one. 
Often, the secoiid strongest was another language common in their native ^ 
country (e.g., Chinese in Malaysia, Hindi for speakers of several Indian 



For the same reason that we asked interviewees how they would explain 
"native country," we also asked them how they would explain "native, language. 
Nearly all the responses fell into one (or more) of three categories: the 
language you first learned (or the language of your childhood), the language 
spoken in your home (or by your parents or f2unily), or the language of your 
native country. 

17 
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Table 3 



Interviewees' Native Languages and Other Languages Spoken 



Native 
language 



Number of 

interviewees giving 
this as their 



native 



anguage 



Number of 

Interviewees saying 
they knew at least^ 
one other language 



Other languages known 




Akan 

Araharic 

Arabic 

Bengali 

Chinese 

Creole 

English 

Far si 

French 

German 

Greek 

Hungarian 

Indonesian 

Japanese 

Konkoni 

Korean 

Malay 

Orlya 

Pape 

Somali 

Spanish 

Tamara 

Tamil 
Telegu 



ICretich 
French , Russian 
Hindi 

German , Japanese, Russian 
French, Portuguese 
French, German, Hindi 
Turkish 

Arabic, Creole, 
Russian, Spanish 

French, Italian 

French, Italian 



German 

French, German 

Japanese 

Chinese 

Assamese, Bengali, Hindi 

Cteole , Portuguese , 
tribal languages , 



Fr en c h , H e br e v , 1 1 a 1 tan 

Afrikaans, German, 
tribal languages 

German , Sinhalese 

French, German, Hindi, 
Kannada 
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Table 3 (Continued) 
Interviewees' Native .Languages and Other Languages Spoket 



Number of ^ Number of 

interviewees giving interviewees saying 
Native this as their * they knew at least^ 
language native language- one other language 



0t>4r languages known 




Thai 

Turkish 

Twi 

Vietnamese 
Yoruba 

Totals 



Chinesfe, German 



French, German 
Chinese 

French, Spanish 



^Names of languages are those given b/ respondents. Some of these names are not 
in the TOEFL Handbook for examinees 

^Excluding English and language^currently being studied, and without regard to 
degree pf fluency. 



/ 
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languages }y and interviewees had learned, it virtually simultaneously with 
their native language. ^ 

Two other variables were dropped because they seemed neither reliable 
nor valid* Interviewees had coinsiderable difficulty naming for us the last 
educational document or credential they had received* The name was often 
in a foreign language for which they did not know an equivalent term in 
English. * In some cases it appeared that they were not even sure that they 
had received a document from the last institution they attended; or if they 
hady exactly what it was called. Interviewees also had difficulty recalling 
the number of hours per week they had spent in their* last English class » 
and it was clear from the way th6y responded that they were giving us only 
vague approximations* The number of hours may well have varied over the 
last few years of school, or even within the school year, and some interviewed 
had evidently not studied English as a separate subject at all during at 
least their last year of school-. (THiis included some for whom English was 
the language Qf instruction, and te classify them as having had no hours of 
English^^^ring the last year would obviously be misleading. } This variable 
had been included in the belief that number of hours per week might be a 
better predictor of language achievement than number of years in which 
English-language courses had been taken. However, what happens during 
those hours is probably even more important and is still highly variable. 
Consequently, number of hours may not really contribute much^ to the interpre- 
tation of a score beyond -what is already known from knowing number of 'years* 
Finally, number of hours is more likely to be standardized within the educa* 
tional system of a given country than is number of years (i.e., students 
may elect to study English, or another foreign language, and to study it 
for a varying number of years | but once they have made their choice f or^ a 



20 



given year, the curriculum for that year will, in tnany places, be more or 
less prescribed) I so that' future research may be alble to use number of 
years of study of English together with native country to achieve 
more precise measunement of the amount and quality of instruction in 
English that students have had. 

Exploration of Relationshipj/ ^ ^ 

The screening process left us with nine new variables whose aspociation 

with TOEFL scores could be explored,' Because of the purpose of this study 

and the small size and non-random nature of the sample, the use* of powerful 

* * J- 

and rigorous statistical tools for the testing of hypotheses was deemed both 

unnecessary and inappropriate. Instead, we made use mainly of the techniques 

of exploratory data analysis (Hartwig, 1979; Tukey, 1977), Which rely heavily 

on such relatively robust measures as the median and the interquartile range 

(or "mldspread") to describe variables and on median traces (analagous to 

regression lines) to examine relationships between variables. Only after 

these preliminary explorations had been completed did we proceed to 

regression analysis. 

Before the effects of any of the new variables on TOEFL scores could be 

studied, it was necessary to consider the likelihood that mi^ny, even most, 

of them would be related to the inverviewees' educational status — i.e., 

whether they were in the U.S. as undergraduates or as graduate students — 

atid that educational status would in turn be related to TOEFL score. If 

this were the case, then variables that appeared to be related to TOEFL 

score might be so only as an artifact of their simultaneous relationship 

to educational status. The median score for the 33 undergraduates for whom 



2i 



-19- 

we had scores was 510; for the 17 graduate students, 580 (with midspreads 

of 66 and 87, respectlve.ly) — obviously a difference large enough to require 

13 

that educational status 'be used as a control In all further analysis* 
The distributions of these undergraduate and graduate-student Interviewees 
on the nine new variables, and the median scores for persons In each 
category, are shown In Tables 4 and 5. 

Three of the new variables were qualitative: the type of education^ 
Institution previously attended In the U.S., whether the Interviewee had 
had a Job where English was used, and whether any brq^^hers or sisters had 
attended school outside their native country. None of these s^ems. to be 
a very promising variable. Variation by type of U.S. Institution attended 
was small and Irregular. Among undergraduates, the highest median score was 
attained by those who had not attended a U.S. Institution* at all — a result 
that Is contrary to what would reaspnably be expected', .though It may reflect 
a relatively greater degree of selectivity In the adoilsslon of applicants 
In this category. The dlffereff^s between Interviewees who had held 
a Job where English wds used and those who had not were small and In 
opposite directions for undergraduates and graduate students. It might 
be thought that the effectiveness of this variable coqld> be Improved by 
asking not merely whether candidates had a Job where English was used but 
also how much English was used and/or for how long the candidate had had 
such Jobs — but it is unlikely that ^he answers to these additional probes 



13 

Within educational status, median scores of men and women were virtually 
identical: 511.5 and 510, respectively, among undergraduates; and 580 and ^ 
575 among graduate students* 

14 

Using data for all degree-seeking candidates who took TPEFL 
between 1977 and 1979 (N - 235,738), Wilson (forthcoming, (aj) made the 
slBilar-*-*and equally surprising-^f Ind^ng that candidates who took the 
test at a testing center abroad tended to earn higher scores than those 
who took it in the U.S. * 



ERIC 



2Z 



' -20- 
Table 4 

Distributions and TOEFL Scores of Undergraduate and Graduate-Student 
Interviewees' on Four Potentially Useful Qualitative Variables 



Number of : 



Under- 
graduates 



Graduate 
students 



Median score of ; 
Under- Graduate 
gra4uates students 



Type of U.S. educational 
Institution previously 
attended 

None 

Secondary school 

ESL program 

College or university 

Whether respondent had had a 
J9b where English was usdd 

Y^s 

No 

No answer 

Whether brothers or sisters 
had attended school abroad 

Yes 

No 

Read English-language 
newspapers, magazines, or books 

Yes, frequently 

Yes, occasionally 

No 

No answer 



10 
8 
9 
6 



12 
20 ■ 
1 



18 
15 



9 
13 
11 



11 

3 
3 



/7. .8 
8 
1 



5 
12 



7 
5 
4 
1 



560, 
495 
SCO 
501 



508.5 
513 



511.5 
510 



553 
510 
461 



570 

593 
580 



575 
561.5 



547 
591.5 



590 
580 
541.5 



^Respondents who had attended more than one type were classified according to 
the type at the highest level attended. 
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would be very reliable* The difference between those who d^d and did not 
have brothers or sisters who attended school outside their native country ' ^ 
was very small for undergraduates; and while it was quite large aaong 
graduate students y it was in a direction that is contrary to conaon-sense 
expectations* 

The relationship between TOEFL score and the reading of English- 
language newspapers, magazines » or books, however, was clear, strong, 
and plausible* In asking this question, we tried to obtain from the 
interviewee an estimate of the frequency of such reading* Those who said 
they engaged in it three hours a week or more were classified as "frequent" 
readers of English-language materials; those who said.^hey generally spent 
less than thre^ hours a week at it were classified as "occasional" readers* 
However, many interviewees were understandably hesitant about giving an 
answer in terms of hours, but they were almost always able to say whether 
it was something , they did on a^ regular basis (e^g* , "I subscribed to 
Time magazine and read it nearly every week") or something they did only 
sporadically or when an opportunity happened to present itself; the former 
were classed with the "frequent" readers and the latter with the "occasional" 
ones* When the respondents were categorized that way, the. frequent readers 
had higher median scores than (she occasibnal readers, and the occasional 
readers had higher median scores than those who did not read English-language 
materials at all; and this was true among both undergraduates and graduate 
students, with the difference between the highest and the lowest categories 
being 92 points among the former and 48*5 points among the letter* This 



Most of the siblings who. had attended school outside their nat 
country had done so in the D*S* or Great Britain* yr 
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qudBtlon surely commends Itself for consideration as a new item of 
background information. 

/ As a result of the findings up to this point, and in order to facilitate 
f.urther analysis , the^TOEFL scores were re-expressed in the following way: 
Interviewees w^re categorized according to their educational status and 
Englis^h-language reading simultaneously. They were then given new scores, 
equal to the deviation of their original score from the median score of^ 
.all respondents in their category. 16 Since -ISS^was the largest negative 
deviation of these new scores, 135 was added to each score for the sake of 
eliminating negative scores. Finally, in order to bring the distribution 
of ,the scores closer to symmetry (and thus minimize the possibility that a 
skewed distribution would obscure relationships between TOEFL scores and 
other variables), they were transformed by the equation 

RTOEFL -/dTOEFL 20d\2, 



ml DTOEFL 20d\ i 
V 20 I 



where DTOEFL is the deviation plus 135 points and RTOEFL is the new score 
to be used in the remainder of the analysis. (The numbers 200 and 20 
were chosen because they seemed to maximize the symmetry of the resulting 
distribution. Further details on this transformation are given in Appendix C.) 
RTOEFL ranges from 100 to 486, with a median of 281 and a midspread of 
147. The effect of this transformation is that educational status and 
frequency of reacting English-language materials no longer account for any 
differences among interviewees' scores that we may find. 



16The one interviewee — ^a graduate student — who did not give us 
sufficient information about . reading of English-language materials was 
assigned a score equal to the deviation of his score from the median for 
all graduate students. 
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The distribution and RTOEFL scores of the interviewe'es on £he five 

quantitative variables are given in Table 5. The^fact that number of years 

of education (prior to earning the moat .recent TOEFL acor^) waa unrelated t 

RTOEFL, score except at the extremes is not surprising, since It j^a chiefly 

a function of educational status: The median number of yeati for 

undergraduates was 13 and for graduate students 17, with mid^reads of 

only 3 and A years, respectively. The median trace of RTOEFL score against 

t;rhi 8 variable produced an essentially flat line. Thus, this variable would 

t 

seem to provide little /interpretive help beyon^ what could be 'derived frow^^^^ 
educational status. 

Total number of years of formal study of Ertglish, on the other hand, 
appears tp be a quite promising variable. It showed a steady and 
substantial rise in scores between each pair of intervals except the first. 
Examination of the scatterplot of these two variables against fhe regression 
line of their relationship (estii^ated RTOEFL « 228.8 + 6.03 [number of years 
of study of English]) revealed one clear outlier: a man who had studied 
English for only 1.5 years (in an ESL program in the U.S.) but h*d attained 
a TOEFL score. Of 600 (RTOEFL « 387). This man was among the oldest of our 
interviewees and was evidently a person of very unusual abilities. He had 
already mastered three other langyages in, addition to his native language. 




17 

In this and all other instances, the median trace used intervals finer 
than those shown in Table 5 (which were set only for convenience of 
presentation) and employed the smoothing techni«]ues described in Hartvig 
(1979) to minimize the effects of small numbers of cases in each' interval. 
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. Table 5 

Distributions and RTOEFL Scores of Interviewees 
on Five Potentially Useful Quantitative Variables 



Number of years of prior education 

less than 12 

■12-13 

14-15 

16-17 ■ 
* 18 or mpre ' . 

. ■■' , * ' ■ 

Number of years of formal study of English 

1-A 
5-8 
9-12 
13-16 

17 or more . ^ 

Number of years qf schooling In which 
English was language o£ Instruction 



0.0 


- 0.9 


1.0 


-1.9 


2.0 


- 2.9 , 


3.0 


- 3.9 


A.O' 


- A. 9 


5.0 


or more 



Number of years of father's education 



0-A 

5-9 - 

10-lA 

15-19 

20 or more 
No answer 



Number of years of mother's education 



0-A 
5-9* 
ib-lA , 
15-19 • ' 
20 or more 
No answer 



5 
15 

9 
11 
10 



15 - 
12 
10'' 
A 



17 
9 
5 
1 
7 

11 



6 

9 
11 
16 

A 

- A 



8 
13 
11 
14 

1 
. 3 



Median 
RTOEFL score 



25A' 
28A 
281 
281 
303 



281, 
'26A 
281. 
292 
AlO 



276 
289 
28A 
216 
303 
281 



31A.5 

25A 

281 

286.5 

311 



252 

25A. 

281 

311.5' 

28A 
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had received a t^ch^lor's and a master's degree in a country where his * 
native language was not spoken, and had held a highly rej&ponsi^e posit loii 
aJs a civil servant In his government. In view of the£^0 rather ap^clai 
clrcumlstancep, he was excluded from the rest of the andlys|.a: 
Included In the other parts of Table 5). With that exclusion^ the median 
RTOEFL score for persons with 1-4 years of fdrml;€^^ was 
reduced to 269.5. ^ ' • • 

The number of year^ of formal study of :^Englll8h;^^^^^ years during 

which English was tlie» language of Instruction^ but^^ t^^ of 
the latter kind was also looked at as a separate Variable on the ground 
that each such year might contribute more tov a/ per^ott^^ oi English 

th^n a year in which English was simply one subject of /study among many.' 
The trend of scores shown in Table 5 does not give much support to that idea. 
(Elimination of the interviewee mentioned above improved the picture 
somewhat: The median RTOEFL score in the category of persons with 1.0-1.9 
years during which English was the language of instruction dropped to 276.5.) 
We experimented with various ways of devising and combining two measures 
of study of Eijglish-Te.g. / using number of years of schooling in which 
English was the language of instruction and number of years of other 
formal study of English, giving double weight to each year in which English 

was the language of instruction, re-expressing the measures in an effort to 

/I 

give them greater symmetry — but none of these manipulations produced a 
stronger relationship with RTOEFL than. simply th^ straightforward measure, 
total number of years of formal study of English* Nevertheless, it should 
be remembered that the sample we were working with is small and' non-random. ' 

It is possible that, with a larger and more representative sample of TOEFL 

t 

, 28 
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• . 

Candidates^ two measures of the amount or type of study of English might be 
found that would be substantially more helpful in interpreting the scores 
than simply total number of /ears of formal study. It does seem prudent, 
however, to be sure that, whatever questions are asked, the answers make 
it possible to obtain a measure of total number of years of formal study. 

Number of years of father's and mother's schooling was included as a 
measure of socioeconomic background or status (SES), one of the most 
fundamental variables in educational and social-science research because of 

18 

its pervasive effects on every aspect of human development and social life. 
Parental occupation, rather than education^ is perhaps the si^le most widely 
used index of SES, but its use in the TOEFL context seemed questionable. 
Occupational structures vary considerably among different nations, 

19 

especially among nations at different stages of economic development. 
Furthermore, it would be quite difficult to frame a question about parental 
occupational status that would be clear enough to assure useful responses, 
if only because the English words used to translate even the "same" 
occupation differ from one country to another. A detailed list of occupations 
would not be practicable, yet a ^brief list of occupational categories would 
leave uncertainties about the content of each. Finally, when occupational 



^^Fbr a sumujary of the literature on these effects, see Vanfossen 
(1979). X 

19 

For a vivid illustration of the magnitude of the problems involved 
in establishing occupational categories that are comparable across countries, 
see Treiman (1977). V/hile Treiman achieved remarkable results, he used 
data from only 51 countries, whereas TOEFL candidates come from about 150, 
and on some countries he had information on as f^ as a dozen occupations, 
so his problem was relatively "small" — yet his description of the 
methodological difficulties he had to deal with makes it clear that it would 
be prohibitively expensive to do anything similar on a regular basis. 
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status Is used as a measure of SES, there Is the troublesoae problem of 
the mother* a status: If It is not asked for, It raises questions about 
equity and may also mean a loss of valuable Information; yet If It Is asked 
for, difficult decisions have to be made about classifying mothers who do 
not have paid employment (not to mention the Issue of cross-cultural 
differences In the sex compositions of occupations). Use of family Income , 
as an SES measure would have encountered even thornier problems of currency 
equivalents, , the significance of family size and age composition, and the 
varying meaning of cash Income. 

The use of parental education avoids many of these difficulties. 
"Schooling** Is a widely jrecognlzed English word, with reasonably similar ' 
meaning everywhere, and a year Is a universally accepted unit for 
measuring how much of It one has had. The question needs neither a detailed 

checklist nor a set of summary categories; It can be asked open-ended, 

« 

^ since the use of numbers Is, again, universal. It Is possible for both 
father and mother to have It, and to have It In Independent amounts, and 
Zero years has an Interpretable meaning for both of them. None of our 
Interviewees had any difficulty In understanding our question; a few said 
they could not remember, but that would probably be as much of a problem 
with any other SES question. 

Of course, there remains the question of whether father's or mother's 
education can contribute anything to an explanation of TOEFL scores beyond 
what would be gained from the use of the other variables that we have 
considered so far. Table 5 does Indicate that the education of each parent 
was clearly, though not perfectly, related' to RTOEFL scores. (Elimination 
of the "outlier** Interviewee discussed above did not materially change these 
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relationships.) RTOEFL scores. It stiould be remembered, are already free 
of the Influence of educational status and reading of English-language 
materials, variables which could well be related to parental education. 
However, It would be reasonable to expect that father's and mother's 

education were related to each other and that they were In turn related In 

'20 

some way to total number of years of formal study of English. 

Father's and mother's education were Indeed related to eaCh other, 

though not so strongly as to suggest that they^ were interchangeable 

2 

variables. The coefficient of determination (R ) of their relationship 

was .525 — i.e., a little more than half of the variation in one could be 

'21 

accounted for by the variation in the other. Neither of them, however, 

2 

Was related to .years of study of English; R for mother's education was 
zero, and for father's .005. What all of this suggests is that the three 
variables have independent importance. 

Further analysis confirmed that conclusion, and yielded one interesting 
and unexpected finding. Years of study of English, father's education, and 
mother's education each were related about equally strongly to RTOEFL scores 
the respective coefficients of determination were - .10, .09, and .13. 
When RTOEFL scores were expressed as deviations or residuals from the 
regression of RTOEFL on years of study of English, in order to remove the 



20 

The distributions of parental education shown ^n Table 5 strongly 
Suggest that the parents of TOEFL candidates are disproportionately drawn 
from the well-educated strata of their respective countries, although no 
precise comparison was possible within the scope of this study. 

21 

A median trace of the relationship showed it to be virtually linear, 
so that, the R would not be increased by re-expressing the variables or 
by searching for some sort of nonlinear function. 
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effect of the latter on the former, R for father's education and ^these 

2 

residuals dropped, but only slightly, to ,07, while R fot;, Bother's 
education actually went up somewhat, to •16« Finally, when the effect of 

both years of study of English and father's education were removed 

' 2 
by the same technique, mother's education still showed an R of ,05 — ^a 

Value high enough, especially In view of the nature of this sample, to 

suggest that mother's education should definitely be considered for inclusion 

among the new items of background information to be sought. Indeed, it 

22 

appears to be even more important than fath^r^s education, though it 
would probably seem peculiar and might lose potentially valuable information 

if mother's education were asked for and not father's. 

* - ... I , 
Conclusions 

From the sifting through of some two dozen variables that mlgbt be 
adde4 to the background information collected from TOEFL candidates, four 
variables emerged as being the xaoBt promising: 'total number of years 
of formal study of English; reading of English-language newspapers, books, 
and magazines; number of years of father's education; and number of year» 
of mother's education. Each of these was easily asked fot and readily 
responded to, and each was independently related to TOEFL scores, in the 
interpretation of which they would probably find their most Immediate use. 

^^This was not because, as might be thought, mother's education was a 
more differentiated variable than father's. Mldspreads for, the two were 
nearly equal — 10 and 9.5 years, respectively (and so, for that matter, 
were the standard deviations— 5.77 and 5.68, respectively). 
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Collectively, they offer the promise of enhanced research on various 
aspects of the^ TOEFL candidate population and on foreign students generally.- 
For example, they could be used in studies of the effects oh success in 
college of knowing the language (much formal study but low reading of 
English-language materials at a given level of TOEFL scores) vs. knowing 
the culture (same level of TOEFL score but less formal study and pore 
reading of English-language materials). Parental education, as a measure 
of socioeconomic background, would be important to control in this and 
many other sorts of research, and its own effects and its effects in 
interaction with other variables (e.g., native country) would also be 
important issues for investigation. 

Moreover, this would be an opportune time for introducing new * 
background items. In accordance with a recent decision of the TOEFL 
Policy Council, the number of institutions that candidates may designate 
as score recipients has been reduced from four to three. Each institutional 
designation requires a six-column field: four for the institutional 
identifying code and two for a departmental code (used by graduate 
students). Thus, four of the newly freed columns could be used for 
parental education (two each for father and mother, asked in open-endec^^{^ 
fashion); one for reading of English-language materials (in checklist 

V 

form: e.g., "At least once a week," "Only occasionally," and "Never"); 
and one for years of formal study of English (asked in open-ended foVm * 



^^The information which follows was kindly provided by the TOEFL 
program staff. 



33 



-31- 

but with the lowest and highest responses — e.g., less than 5 years and ' 
more than 13 — collapsed into single categories so that not more than one 
column would be required).^* These questions could be added without 
change In the answer sheet and with only minor other costs, such as the 
one-time cost of changing the candidates' handbook to Include the new 
questions. Since the resulting Information would be processed In ^ 
conjunction with the processing of the rest of the Information on the 
answer sheet » additional processing costs would also be hardly noticeable. 
In short, the opportunity presents Itself for a substantial and valuable 
addltloti to the stock of Information about TOEFL candidates for very 
little expense. 



^^If additional space could be found on the answer sheet, number of 
years of formal study of English might be broken down into number of 
years In which English was studied as a subject and number of years In 
which it was the language of Inst rue tlonr 
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APPENDIX A 



INTERVIEW QUESTIONS FOR STUDENTS WHO HAVE TAKEN TOEFL 

As you probably know,. I am doing a study for the sponsors of the Test of 
English as a Foreign Language. They would like to learn more about the 
students who take their test, so that they can Improve their services to 
students In the future. As part of this study, ve are asking students from 
different countries some questions about their educational background. We 

( 

have tried to make the questions clear, but If any of them are not, please 
stop me and I will try to explain them better. This Intetvlav Is not a 
part of your studies here, and your answers will not affect then In any way. 
In fact, your answers will not be communicated to any person at this (collage, 
university) , and they will not be rep6rted anywhere In any way that will allow 
you to be Identified. So I hope you will do your best to answer every question. 
However, If there Is any question that you do not want to answer, simply tell 
me that and I will go on to the next one. I expect that we will be finished 
In about 30 minutes, but If you have to leave sooner for any reason, you are 
free to do so. Is all of that clear, so thaj: we may begin the Interview? 
[Pause] [Add^ If necessary:] Am I perhaps speaking too fast? [Pause] 
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First, would you tell me whether you are a ^r^uate student or an 
undergraduate. ^ 

And Is this- your first yearY(name of college) ox your second? 



O.K, good. Now, what country were you living in Just before coming 
to ^in£m (name of college)? 



How many y^ars did you attend school in (that country)? 

IF NONE; 1.3.1 What country were you living in before that? 
[Then repeat question 1.3.] 

What type of school was the last one you attended? 

[Probe to find out whether.it was at secondary or postsecondary levels 
and if latter, whether it was university, technical, teacher- training, 
or other type of institution.] 

Did you receive some sort of document upon completing that school? 

IF YES: 1.5.1 What was it called? 

[Get English translation if necessary.] 

Had, you ever attended a school in the United States before coming to 
(name of college)? 

IF YES; 1.6.1 Would you please tell me about that? 

[Find out educational level, length of time, and 
R.'s satisfaction with ability to use English.] 

Have you ever attended school in any country beside (answer to 1.3 
or 1.3.1) and the United States? 

IF YES: 1,?;1 Would you please tell me about that? 

[Find out where, hoy long, and educational level, J 
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1.8 Do you expect Co receive a degree In the U.S.? 

% 

IF YES: What degree, and In what field? 

1.8.2 Do you expect to receive It here, at (name of college), 
- or do you think you sight transfer soae time? 

UNLESS EXPECTED DEGREE IS PH.D. 

1.9 Do you expect to continue your studies at any other college or university, 
here or anywhere else? ^ 
IF YES: [Probe for details.] 



♦ 



ERIC 



39 



ERIC 



2.0 Next, we would like to find out more about how and where students 
from other countries have learned English. Could you tell me, first; 
how many years you studied English In school before coming to the U.S.? 

2.1 In what kind of klnds of schools did you study English? 

[If necessary, explain: primary, secondary, college-level; schools 
operated by the government; schools operated by a religious groups;^ 
schools not operated by either the government or a religious group. 
Record any other characteristics given— e.g. , one-sex or coeducational, 
urban or rural, large or small, etc.] 

2.2. During your last few years in school, about how many hours a week did 
you spend in your English class? 

2.3 Did your teachers speak English in class moat of the time? 

2.4 Dl^l you have any other courses in which the teacher spoke English in 
the classroom most of the time? * * 



2.5 Did you use English-language books in any of your courses— that is, 
beside your English courses? 

2.6 Have you had any Instruction in English here in the U.S.? 
IF YES: [Get details.] 

2.7 Have you had any formal instruction in English beside what you have 
told me about so far— for example, have you had any private tutorlnft 
or attended a school in another country, or listaned to couriis on 
radio or television, or anything else? 



IF 



YES: (Ask for specifics parallel to those in 2^1-2.5.] 



2.8 Have you ever had a Job where you used English? 

IF YES: [Ask for specifics: nature of the Job, how English, was used.] 
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2.3 Does anyone In your- family beside yourself know English? 

IF YES: [Ask for specified: Who they were, Whether they ever used 
English. with R.] * ^ 

2.10 Before coming to the U.S.» did you have any friends ^o knew English?' 
^ YES: .2.10.1 Did you ever speak English with these friends? 

2.11 Before coming to the U.S., did you rJad any English-language 
newspapers, magazines, or books, besides those you read in school? 

IF YES: [Asl^ for some examples and an estimate of frequency.] 

2.12 Were you ever able to listen to English-language radio broadcasts ^ * 
, or television programs or see English-language films? J ^ 

IF YES: [Ask for some examples and an estl^te of frequency.]^ ' 

2.13' Had you ever be6n in the U.S. before coming to this (college, university)? 
[Add, if necessary:'] .. .beside the time you have already told me about? 

IF YES: [Ask when previous vislJrls) took place, how loijg they lasted, 
^ what the nature of the activities were, and how satisfied 

R. was with ability to use English.] 

2.14 Had you ever/'been in any othffr country where English was the language 

spoken by most of the people? , 



IF YES: [Ask what couhtrles, when, how long, what were the activities 
and, if necessary, how satisfied R. was with ability to use 
English.] 



3.0 I believe that you took the Test of English as a Foreign Language » 
or TOEFL, before you enrolled hfere at (name of college). Is that correct? 

IF N0:» 3.0.1 Did you take any test of your knowledge of English? 

\ . ^ ' . ■ 

IF NOs. [Skip section 3 and go on to section 4.} 

IF YES: [Probe to be sure It wasn't TOEFL — e.g., ask 
who made the test, who published It, who sent 
the scores. If you are reasonably sure It was 
TOEFL, ask the rest of the questions In section 3 
' without using tke TOEFL name. If you are not 
It was TOEFL, or are sure It was not TOEFL, skip 
section 3 and go on to section 4.] 

3.1 Did you take It just once or did you take It more than once? 

IF MORE THAN ONCE: 3.1.1 How m^a/^es did you take It? 

3.1.2 Do you remember when those times were? 
[If possible, gkt answer In terms of both 
year and stage of educational career.] 

3.1.3 What were your reiasons for taking It each 
of those times? 

[If necessary, preface the next question with: ] Thinking now about the last 
time you took TOEFL before coming to this (college, university)... 

3.2 Do you think that the test was a good measure of how well you knew 
English at that time? 

IF NO: 3.2.1 Why do you think It wasn't? How do you think the test 
could be made better? 



42 



A-7 



3.3 Would it be all right witli you If I asljced (person who arranged the 
Interview) what your score on the test was? 

IF R. VOLUNTEERS SCORE: O.K., thanks very nruch. [Go on to 3.4.1 

IF NO [I.e., R. declines permission]: 3.3.1 Would you mind giving 

me your score yourself? 

IF YES: [Be sure you have R. *s namer then ask:l 

3.3.2^ Do you. happen to remember what your score was? 

3.4. When you took the TOEFL, you were asked what your "native country" 

was. Do you remember what country you named in answer to that question? 

IF NO: [Ask R. how he/she would answer the question now, and rephrase 
the following questions accordingly.] 

3.5 Are you also a citizen of (that country)? , 

IF YES: 3.5.1 If a student from (answer to 3.4) were to ask you 

what "native country" means , how would you explain it? 
3.5.2 Why did you answer (native country) to the question \ 
. rather than (country of citizenship)? 

3.6 When you took the TOEFL, you were also asked what your "native 
language" was. Do you remember what language you gave in answer to 
that question? 

IF NO: 3.6.1 Well, how would you answer that question if someone 
asked you it now? 

3.7 Again^ if a student from (answer to 3.4) were. to ask you what "native'' 
language" means, how would you explain that? 
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Have you taken any other standardized tests beside the TOEFL either 
before coming to the United States or since you've been here? 

IF YES: 3.8.1 Could you tell me what they were? 

[Probe ta learn reasons for taking.] 

IF NO OR UNSURE: [Ask about s^ectlflc tests— SAT, GRE, ©lAT. 

IF YES to any of these, probe to learn reasons 
for taking.] 
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iNOTE: SECTION 4 MAY BE OMITTED IF INTERVIEW TIME IS GROWING SHORT.] 

t 

4.0 Do you know any languages other than (native language) and English? 

IF YES: 4* 0*1 Which other languages do you know and how did you 
learn them? 

4.1 Which would you say is your strongest language, the one you know 
best and are most comfortable with? 

[IF R. KNOWS MORE THAN TWO LANGUAGES]: 

4.2 Which would you say is your next strongest? 

> 
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5.0 Would you teXl me your main reasons for coming to the United States 
to study rather than studying In your own country or another country? 

5.1 How did you learn aboiit colleges and universities In the U.S.? Where 
did you^get that Information from? ' 

5.2 How many colleges or universities In the UlS. did you apply to? 



3 Why did you decide to come to this particular (college, university)? 



IF NO: ,5.4.1 Why not? 

5.5 If you were talking to students In (R.'s native country), what are 

the two or three most Important things about colleges and universities 
In the U.S. that you would like to tell them? 




5.4 



Do you feel now that you made the right choice? 
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'6.0 This l8 the last set of questions. Could you tell me d little about 
your family? For example, hov many brothers and sisters do you have? 

IF NONE: [Skip. to question 6.1.] 

* 

6.0.1 Has^ any of ,them gone to school In a country other than 
(R.'s native country)? 

IF YES: 6.0.1.1 WbuW you tell me about that? 

[Probe to learn country, level of study, 
and length of time.] 

6.1 What country did your father grow up In? 

Hov many years of schooling has your father had? 
[Probe to learn countries In which father was educated.] 

6.3 Would you say that (that number of) years of schlDollng Is about* the 

same as most men of his agf In (R.'s native country) have had, or more 
than most, or less than most? 

6. A. And your mother — what country did she grow up In? 

6.5 How many years of schooling has she had? 
[Probe to learn countries.] 

6.6 And would you say that (that number of) years Is about the same as 
most women of her age In (R;'s native country), or more than most, or 
less than most? 
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7.0 Those arte all the questions that I have. Is there anything' else about 
your eduqatlon^ your study In the U.S., or the TOEFL that you think 
It Is Important for me to know or that you would like to tell me, 
that we haven *t talked about so far? 



Thank you very much for coming by and taking the time to talk to me. The 
report of the study for which this Interview was done will be ready next 
March or April, and we will send a copy to the (office which arranged the 
Interviews) so that you can read It If you -re Interested. Thanks again, 
and good luck In your studies! > 



} 
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APPENDIX B 

INTERVIEWEES' COMMENTS ON THE TEST OF ENGLISH AS A FOREIGN LANGUAGE 

1 

Partly as a rapport-building preface to inquiring about their 
TOEFL scores, we aslced the interviewees, wherever appropriate, "Do you 
thiriHc that ^tte^Xest was a good measure of how well you knew English at 
that time?" (i.e., at the time the test was taken). Two interviewees had 
not taken TOEFL, and in eight other instances the interview situation did 
not lend itself to asking the question. Of the remaining 47 interviewees^ 
18 simply, said they thought it had been a good measure. The rest were 
asked follow-up questions: "Why do you think it Wasn't [a good test]? 
Hov^do you think the test could be made better?" These 29 interviewees — 
df whom 13 said it was not a good measure and 16 said they were not 
Bure or it was all right in some ways but not in others— made a total of 
53 comments about TOEFL and the experience of taking it. 

Almost half the comments (24) revolved around the difficulty level 
of the test. Six interviewees said the listening comprehension section 
was hard, two singled out the ic<aading comprehension section, four the 
vocabulary, and two simply said the test was hard "in general." Related 
to these comments Were five others that raised objections to the specifically 
"American" nature of the test — in the accent on the tapes for the listening 
comprehension section, the vocabulary of either- the listening or the 
reading comprehension section, or the content of the items. On the other 
hand, ten interviewees said they thought the test was easy, even "ridiculously 
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easy, and four of them mentioned the listening comprehension part as 
having been especially easy*^ 

The second most frequently occurring type of complaint dealt with 
the conditions under which the test was taken (9 interviewees). The 
listening comprehension tapes again came in for criticism; it was said 
that they were hard to hear, because either the player or the tape itself 
was defective, or because there was too much noise in the testing environ- 
ment. A few interviewees felt that there had been inadequate supervision 
in the test room, leading to disorder and (some suspecterf) providing 
opportunities for cheating. , . , 

The r^t of the complaints were directed at a wide variety of 

targets: the test was too general or it was too specific, the answer 

sheet was hard to und/erstand, there was too much emphasis on fine points 

/ 

of grammar, or — reminding us that the U.S. and the rest of the world have 
much in common — the multiple-choice format is too constricting. 



Several interviewees spoke English quite fluently; indeed, as 
Table 3 showed, one even said that Eng^lish was his native language. This 
irdises the question of why they were asked to take TOEFL at all. A typical 
institutional rule is that of the University of Ilaryland, which requires 
a TOEFL score from all applicants except (a) native speakers of English, 
defined as "those born and educated in the U.S., English-speaking Canada, 
United Kingdom, Ireland, Australia, New Zealand, or Commonwealth Carribbean, 
and (b) non-native speakers of English "who have received a tertiary degree* 
in one of those countries or regions. Thus, a TOEFL* score was required 
of applicants from countries like India, Malaysia, Singapore, Israel, and 
Nigeria, where English may have been the language of school instruction 
and even of the family. When asked about this, admissions and foreign- 
student personnel admitted that some applicants argued that having to 
take TOEFL was almost an affront, but their response was that the test 
need not cause anxiety to applicants who knew English well while it did 
protect the institution — and indeed the applicant as well — from a situation 
in which a student would be permitted to enroll only to find that he or 
she was unable to keep up with the academic work because of a lack of 
facility in English — and the applicant was not necessarily a good ju<fge 
of whether that was likely to occur. 
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Appendix C 
DE^VATION OP THE RTOEFL SCORES 

Ab pointed out In the text (p. 22), It was desirable, for purposes 
Of further analysis, to eliminate the effects on TOEFL scores of educa- 
tional status and prlot reading of English-language materials. This 
was accomplished by classifying students according to whether they were 
undergraduates or graduate students , and according to whether they had 
been frequent readers of English-language materials before taking TOEFL, 
or occasional readers, or non-readers; determining the median for each 
of the six resulting categories; and calculating a new score, DTOEFL, 
which was the algebraic difference between a score and the median of 
the category In which It fell, plus 135 (this latter for the sake of 
avoiding negatiye scores) • Because this procedure expressed each score 
as the deviation from one of six "local" medians, the result was a 
narrow and highly peaked distribution; DTOEFL scores ranged from 0 to 
241 (compared to the TOEFL range, In this sample, of 375 to 670, or 295 
points), and almost half the cases (22) were between 100 and 150. 
Furthermore, the scores were skewed toward the lower end of the range; 
there were 106 points between the median of the total distribution 
(which was, of course, 135) and the highest score, but 135 points 
between the median and the lowest score. 

Because a distribution departing that far from normality could 
distort or conceal relationships between the scores and other variables. 
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the PTOEFL scores were trafilsfomed Into RTOEFL scores, t>y the formula 

RTOEFL - D?raEFL+200? 

20 . - 

• ' . / 

The terWs In th'ls foraula have no intrinale significance J they Were 
chosen because ^they seemed to be most effective In achieving .normality 
of the sc^re distribution* * , . 

The RioEFL scores range from 100 to 486, a range that is 145 points 
greater thari that of the DTOEFL scores v The maximum number of cases 
in any single 50-point interval is reduced to 18, and the distances 
betwien the median (281) and the extreme scores are more nearly equal 
(1^1 and* 205) • The distribution of RTOEFL scores is thus a better , 
approximation to normality • 

A complete rOster of TOEFL, DTOEFL, and RTOEFL scores for the sampl 
is given in Table C-1 . 
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TABLE C-1 

r 



TOEFL» DTOEFLy and RTOEFL Scores of Undergraduate and Graduate Student 
Interviewees by Frequency of Prior Beading of English-Language Materials 



TOEFL ^ DTOEFL RTOEFL 



UNDERGRADUATES 
Frequent Readers 

✓ 

653 235 473 

630 212 424 

617 199 398 

600 182 365 

553 135 281 

513 ^5 218 

507 89 209 

480 62 172 

467 '49 155 

Median 553 135 281 



» Occasional Readeri 

600 225 452 

533 158 320 

523 148 303 

520 145 298 

520 145 298 

515 140 289 

♦ 510 135 281 

510 135 . 281 

507 132 276 

500 125 264 

500 125 264 

495 120 256 

375 0 100 

Median 510 135 281 
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TABLE C-1 (continued) 



TOEFL 



DTOEFL 



RTOEFL 



567 
540 
520 
513 
463 
461 
447 
445 
425 
420 
377 

Median 461 



Non-Reederi 

241. 
214 
194 
187 
137 
35 
1 

119 



99 
94 
Si 

135 



486 
428 
388 
374 
284 
281 
258 
254 
224 
216 
158 

281 



GRADUATE STUDENTS- 
Frequent Reedefi 



670 
663 
603 
590 
513 
480 
467 

Median 590 



215 
208 
148 
135 

58 

25 ' 

12 

135 



431 
416 
303 
281 
166 
127 
112 

281 



593 
580 
580 
547 
543 

Median 580 



Occasional Readers 

146 
" 135 
135 
102 
98 

135 



299 
281 
281 
228 
222 

281 
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TABLE C-1 (continued) 



TOEFL 



DTOEFL 



RTOEFL 



600 
570 
513 
470 

Median 541.5 



Non-Readers 

' 193.5 
163.5 
106.5 
63.5 

135.0 



387 
330 
235 
174 

281 



647 



No Information about reading 
202 



404 
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